Soft phenotyping for sepsis via EHR time-aware soft clustering.

OBJECTIVE: Sepsis is one of the most serious hospital conditions associated with high mortality. Sepsis is the result of a dysregulated immune response to infection that can lead to multiple organ dysfunction and death. Due to the wide variability in the causes of sepsis, clinical presentation, and the recovery trajectories, identifying sepsis sub-phenotypes is crucial to advance our understanding of sepsis characterization, to choose targeted treatments and optimal timing of interventions, and to improve prognostication. Prior studies have described different sub-phenotypes of sepsis using organ-specific characteristics. These studies applied clustering algorithms to electronic health records (EHRs) to identify disease sub-phenotypes. However, prior approaches did not capture temporal information and made uncertain assumptions about the relationships among the sub-phenotypes for clustering procedures. METHODS: We developed a time-aware soft clustering algorithm guided by clinical variables to identify sepsis sub-phenotypes using data available in the EHR. RESULTS: We identified six novel sepsis hybrid sub-phenotypes and evaluated them for medical plausibility. In addition, we built an early-warning sepsis prediction model using logistic regression. CONCLUSION: Our results suggest that these novel sepsis hybrid sub-phenotypes are promising to provide more accurate information on sepsis-related organ dysfunction and sepsis recovery trajectories which can be important to inform management decisions and sepsis prognosis.

Machine Learning Assisted Discovery of Novel Predictive Lab Tests Using Electronic Health Record Data.

Epidemiological studies identifying biological markers of disease state are valuable, but can be time-consuming, expensive, and require extensive intuition and expertise. Furthermore, not all hypothesized markers will be borne out in a study, suggesting that higher quality initial hypotheses are crucial. In this work, we propose a high-throughput pipeline to produce a ranked list of high-quality hypothesized marker laboratory tests for diagnoses. Our pipeline generates a large number of candidate lab-diagnosis hypotheses derived from machine learning models, filters and ranks them according to their potential novelty using text mining, and corroborate final hypotheses with logistic regression analysis. We test our approach on a large electronic health record dataset and the PubMed corpus, and find several promising candidate hypotheses.

Machine Learning Algorithm Predicts Cardiac Resynchronization Therapy Outcomes: Lessons From the COMPANION Trial.

BACKGROUND: Cardiac resynchronization therapy (CRT) reduces morbidity and mortality in heart failure patients with reduced left ventricular function and intraventricular conduction delay. However, individual outcomes vary significantly. This study sought to use a machine learning algorithm to develop a model to predict outcomes after CRT. METHODS AND RESULTS: Models were developed with machine learning algorithms to predict all-cause mortality or heart failure hospitalization at 12 months post-CRT in the COMPANION trial (Comparison of Medical Therapy, Pacing, and Defibrillation in Heart Failure). The best performing model was developed with the random forest algorithm. The ability of this model to predict all-cause mortality or heart failure hospitalization and all-cause mortality alone was compared with discrimination obtained using a combination of bundle branch block morphology and QRS duration. In the 595 patients with CRT-defibrillator in the COMPANION trial, 105 deaths occurred (median follow-up, 15.7 months). The survival difference across subgroups differentiated by bundle branch block morphology and QRS duration did not reach significance (P=0.08). The random forest model produced quartiles of patients with an 8-fold difference in survival between those with the highest and lowest predicted probability for events (hazard ratio, 7.96; P<0.0001). The model also discriminated the risk of the composite end point of all-cause mortality or heart failure hospitalization better than subgroups based on bundle branch block morphology and QRS duration. CONCLUSIONS: In the COMPANION trial, a machine learning algorithm produced a model that predicted clinical outcomes after CRT. Applied before device implant, this model may better differentiate outcomes over current clinical discriminators and improve shared decision-making with patients.

Improving breast cancer risk prediction by using demographic risk factors, abnormality features on mammograms and genetic variants.

The predictive capability of combining demographic risk factors, germline genetic variants, and mammogram abnormality features for breast cancer risk prediction is poorly understood. We evaluated the predictive performance of combinations of demographic risk factors, high risk single nucleotide polymorphisms (SNPs), and mammography features for women recommended for breast biopsy in a retrospective case-control study (n = 768) with four logistic regression models. The AUC of the baseline demographic features model was 0.580. Both genetic variants and mammography abnormality features augmented the performance of the baseline model: demographics + SNP (AUC =0.668), demographics + mammography (AUC =0.702). Finally, we found that the demographics + SNP + mammography model (AUC = 0.753) had the greatest predictive power, with a significant performance improvement over the other models. The combination of demographic risk factors, genetic variants and imaging features improves breast cancer risk prediction over prior methods utilizing only a subset of these features.

Mixed Approach Retrospective Analyses of Suicide and Suicidal Ideation for Brand Compared with Generic Central Nervous System Drugs.

INTRODUCTION: Several different types of drugs acting on the central nervous system (CNS) have previously been associated with an increased risk of suicide and suicidal ideation (broadly referred to as suicide). However, a differential association between brand and generic CNS drugs and suicide has not been reported. OBJECTIVES: This study compares suicide adverse event rates for brand versus generic CNS drugs using multiple sources of data. METHODS: Selected examples of CNS drugs (sertraline, gabapentin, zolpidem, and methylphenidate) were evaluated via the US FDA Adverse Event Reporting System (FAERS) for a hypothesis-generating study, and then via administrative claims and electronic health record (EHR) data for a more rigorous retrospective cohort study. Disproportionality analyses with reporting odds ratios and 95% confidence intervals (CIs) were used in the FAERS analyses to quantify the association between each drug and reported suicide. For the cohort studies, Cox proportional hazards models were used, controlling for demographic and clinical characteristics as well as the background risk of suicide in the insured population. RESULTS: The FAERS analyses found significantly lower suicide reporting rates for brands compared with generics for all four studied products (Breslow-Day P < 0.05). In the claims- and EHR-based cohort study, the adjusted hazard ratio (HR) was statistically significant only for sertraline (HR 0.58; 95% CI 0.38-0.88). CONCLUSION: Suicide reporting rates were disproportionately larger for generic than for brand CNS drugs in FAERS and adjusted retrospective cohort analyses remained significant only for sertraline. However, even for sertraline, temporal confounding related to the close proximity of black box warnings and generic availability is possible. Additional analyses in larger data sources with additional drugs are needed.

Comparison of Generic-to-Brand Switchback Rates Between Generic and Authorized Generic Drugs.

STUDY OBJECTIVE: Generic drugs contain identical active ingredients as their corresponding brand drugs and are pharmaceutically equivalent and bioequivalent, whereas authorized generic drugs (AGs) contain both identical active and inactive ingredients as their corresponding brand drugs but are marketed as generics. This study compares generic-to-brand switchback rates between generic and AGs. DESIGN: Retrospective cohort study. DATA SOURCE: Claims and electronic health record data from a regional U.S. health care system. PATIENTS: The full cohort consisted of 5542 unique patients who received select branded drugs during the 6 months prior to their generic drug market availability (between 1999 and 2014) and then were switched to an AG or generic drug within 30 months of generic drug entry. For these patients, 5929 unique patient-drug combinations (867 with AGs and 5062 with generic drugs) were evaluated. MEASUREMENTS AND MAIN RESULTS: Ten drugs with AGs and generics marketed between 1999 and 2014 were evaluated. The date of the first generic prescription was considered the index date for each drug, and it marked the beginning of follow-up to evaluate the occurrence of generic-to-brand switchback patterns over the subsequent 30 months. Switchback rates were compared between patients receiving AGs versus those receiving generics using multivariable Cox proportional hazards models, controlling for individual drug effects, age, sex, Charlson Comorbidity Score, pre-index drug use characteristics, and pre-index health care utilization. Among the 5542 unique patients who switched from brand to generic or brand to AG, 264 (4.8%) switched back to the brand drug. Overall switchback rates were similar for AGs compared with generics (hazard ratio [HR] 0.86, 95% confidence interval [CI] 0.65-1.15). The likelihood of switchback was higher for alendronate (HR 1.64, 95% CI 1.20-2.23) and simvastatin (HR 1.81, 95% CI 1.30-2.54) and lower for amlodipine (HR 0.27, 95% CI 0.17-0.42) compared with the other drugs evaluated. CONCLUSION: Overall switchback rates were similar between AG and generic drug users, indirectly supporting similar efficacy and tolerability profiles for brand and generic drugs. Reasons for differences in switchback rates among specific products need to be explored further.

Comparing Mammography Abnormality Features to Genetic Variants in the Prediction of Breast Cancer in Women Recommended for Breast Biopsy.

RATIONALE AND OBJECTIVES: The discovery of germline genetic variants associated with breast cancer has engendered interest in risk stratification for improved, targeted detection and diagnosis. However, there has yet to be a comparison of the predictive ability of these genetic variants with mammography abnormality descriptors. MATERIALS AND METHODS: Our institutional review board-approved, Health Insurance Portability and Accountability Act-compliant study utilized a personalized medicine registry in which participants consented to provide a DNA sample and to participate in longitudinal follow-up. In our retrospective, age-matched, case-controlled study of 373 cases and 395 controls who underwent breast biopsy, we collected risk factors selected a priori based on the literature, including demographic variables based on the Gail model, common germline genetic variants, and diagnostic mammography findings according to Breast Imaging Reporting and Data System (BI-RADS). We developed predictive models using logistic regression to determine the predictive ability of (1) demographic variables, (2) 10 selected genetic variants, or (3) mammography BI-RADS features. We evaluated each model in turn by calculating a risk score for each patient using 10-fold cross-validation, used this risk estimate to construct Receiver Operator Characteristic Curve (ROC) curves, and compared the area under the ROC curve (AUC) of each using the DeLong method. RESULTS: The performance of the regression model using demographic risk factors was not statistically different from the model using genetic variants (P = 0.9). The model using mammography features (AUC = 0.689) was superior to both the demographic model (AUC = .598; P < 0.001) and the genetic model (AUC = .601; P < 0.001). CONCLUSIONS: BI-RADS features exceeded the ability of demographic and 10 selected germline genetic variants to predict breast cancer in women recommended for biopsy.

Human pluripotent stem cell-derived neural constructs for predicting neural toxicity.

Human pluripotent stem cell-based in vitro models that reflect human physiology have the potential to reduce the number of drug failures in clinical trials and offer a cost-effective approach for assessing chemical safety. Here, human embryonic stem (ES) cell-derived neural progenitor cells, endothelial cells, mesenchymal stem cells, and microglia/macrophage precursors were combined on chemically defined polyethylene glycol hydrogels and cultured in serum-free medium to model cellular interactions within the developing brain. The precursors self-assembled into 3D neural constructs with diverse neuronal and glial populations, interconnected vascular networks, and ramified microglia. Replicate constructs were reproducible by RNA sequencing (RNA-Seq) and expressed neurogenesis, vasculature development, and microglia genes. Linear support vector machines were used to construct a predictive model from RNA-Seq data for 240 neural constructs treated with 34 toxic and 26 nontoxic chemicals. The predictive model was evaluated using two standard hold-out testing methods: a nearly unbiased leave-one-out cross-validation for the 60 training compounds and an unbiased blinded trial using a single hold-out set of 10 additional chemicals. The linear support vector produced an estimate for future data of 0.91 in the cross-validation experiment and correctly classified 9 of 10 chemicals in the blinded trial.

Optimization of reproductive management programs using lift chart analysis and cost-sensitive evaluation of classification errors.

The common practice on most commercial dairy farms is to inseminate all cows that are eligible for breeding, while ignoring (or absorbing) the costs associated with semen and labor directed toward low-fertility cows that are unlikely to conceive. Modern analytical methods, such as machine learning algorithms, can be applied to cow-specific explanatory variables for the purpose of computing probabilities of success or failure associated with upcoming insemination events. Lift chart analysis can identify subsets of high fertility cows that are likely to conceive and are therefore appropriate targets for insemination (e.g., with conventional artificial insemination semen or expensive sex-enhanced semen), as well as subsets of low-fertility cows that are unlikely to conceive and should therefore be passed over at that point in time. Although such a strategy might be economically viable, the management, environmental, and financial conditions on one farm might differ widely from conditions on the next, and hence the reproductive management recommendations derived from such a tool may be suboptimal for specific farms. When coupled with cost-sensitive evaluation of misclassified and correctly classified insemination events, the strategy can be a potentially powerful tool for optimizing the reproductive management of individual farms. In the present study, lift chart analysis and cost-sensitive evaluation were applied to a data set consisting of 54,806 insemination events of primiparous Holstein cows on 26 Wisconsin farms, as well as a data set with 17,197 insemination events of primiparous Holstein cows on 3 Wisconsin farms, where the latter had more detailed information regarding health events of individual cows. In the first data set, the gains in profit associated with limiting inseminations to subsets of 79 to 97% of the most fertile eligible cows ranged from $0.44 to $2.18 per eligible cow in a monthly breeding period, depending on days in milk at breeding and milk yield relative to contemporaries. In the second data set, the strategy of inseminating only a subset consisting of 59% of the most fertile cows conferred a gain in profit of $5.21 per eligible cow in a monthly breeding period. These results suggest that, when used with a powerful classification algorithm, lift chart analysis and cost-sensitive evaluation of correctly classified and misclassified insemination events can enhance the performance and profitability of reproductive management programs on commercial dairy farms.

Cone structure in subjects with known genetic relative risk for AMD.

PURPOSE: Utilize high-resolution imaging to examine retinal anatomy in patients with known genetic relative risk (RR) for developing age-related macular degeneration (AMD). METHODS: Forty asymptomatic subjects were recruited (9 men, 31 women; age range, 51 to 69 years; mean age, 61.4 years). Comprehensive eye examination, fundus photography, and high-resolution retinal imaging using spectral domain optical coherence tomography and adaptive optics were performed on each patient. Genetic RR scores were developed using an age-independent algorithm. Adaptive optics scanning light ophthalmoscope images were acquired in the macula extending to 10 degrees temporal and superior from fixation and were used to calculate cone density in up to 35 locations for each subject. RESULTS: Relative risk was not significantly predictive of fundus grade (p = 0.98). Only patients with a high RR displayed drusen on Cirrus or Bioptigen OCT. Compared to an eye with a grade of 0, an eye with a fundus grade equal to or greater than 1 had a 12% decrease in density (p < 0.0001) and a 5% increase in spacing (p = 0.0014). No association between genetic RR and either cone density (p = 0.435) or spacing (p = 0.538) was found. Three distinct adaptive optics scanning light ophthalmoscope phenotypical variations of photoreceptor appearance were noted in patients with grade 1 to 3 fundi. These included variable reflectivity of photoreceptors, decreased waveguiding, and altered photoreceptor mosaic overlying drusen. CONCLUSIONS: Our data demonstrate the potential of multimodal assessment in the understanding of early anatomical changes associated with AMD. Adaptive optics scanning light ophthalmoscope imaging reveals a decrease in photoreceptor density and increased spacing in patients with grade 1 to 3 fundi, as well as a spectrum of photoreceptor changes, ranging from variability in reflectivity to decreased density. Future longitudinal studies are needed in genetically characterized subjects to assess the significance of these findings with respect to the development and progression of AMD.

Childhood asthma clusters and response to therapy in clinical trials.

BACKGROUND: Childhood asthma clusters, or subclasses, have been developed by computational methods without evaluation of clinical utility. OBJECTIVE: To replicate and determine whether childhood asthma clusters previously identified computationally in the Severe Asthma Research Program (SARP) are associated with treatment responses in Childhood Asthma Research and Education (CARE) Network clinical trials. METHODS: A cluster assignment model was determined by using SARP participant data. A total of 611 participants 6 to 18 years old from 3 CARE trials were assigned to SARP pediatric clusters. Primary and secondary outcomes were analyzed by cluster in each trial. RESULTS: CARE participants were assigned to SARP clusters with high accuracy. Baseline characteristics were similar between SARP and CARE children of the same cluster. Treatment response in CARE trials was generally similar across clusters. However, with the caveat of a smaller sample size, children in the early-onset/severe-lung function cluster had best response with fluticasone/salmeterol (64% vs 23% 2.5× fluticasone and 13% fluticasone/montelukast in the Best ADd-on Therapy Giving Effective Responses trial; P = .011) and children in the early-onset/comorbidity cluster had the least clinical efficacy to treatments (eg, -0.076% change in FEV1 in the Characterizing Response to Leukotriene Receptor Antagonist and Inhaled Corticosteroid trial). CONCLUSIONS: In this study, we replicated SARP pediatric asthma clusters by using a separate, large clinical trials network. Early-onset/severe-lung function and early-onset/comorbidity clusters were associated with differential and limited response to therapy, respectively. Further prospective study of therapeutic response by cluster could provide new insights into childhood asthma treatment.

Assessment of genetic and nongenetic interactions for the prediction of depressive symptomatology: an analysis of the Wisconsin Longitudinal Study using machine learning algorithms.

OBJECTIVES: We examined depression within a multidimensional framework consisting of genetic, environmental, and sociobehavioral factors and, using machine learning algorithms, explored interactions among these factors that might better explain the etiology of depressive symptoms. METHODS: We measured current depressive symptoms using the Center for Epidemiologic Studies Depression Scale (n = 6378 participants in the Wisconsin Longitudinal Study). Genetic factors were 78 single nucleotide polymorphisms (SNPs); environmental factors-13 stressful life events (SLEs), plus a composite proportion of SLEs index; and sociobehavioral factors-18 personality, intelligence, and other health or behavioral measures. We performed traditional SNP associations via logistic regression likelihood ratio testing and explored interactions with support vector machines and Bayesian networks. RESULTS: After correction for multiple testing, we found no significant single genotypic associations with depressive symptoms. Machine learning algorithms showed no evidence of interactions. Naïve Bayes produced the best models in both subsets and included only environmental and sociobehavioral factors. CONCLUSIONS: We found no single or interactive associations with genetic factors and depressive symptoms. Various environmental and sociobehavioral factors were more predictive of depressive symptoms, yet their impacts were independent of one another. A genome-wide analysis of genetic alterations using machine learning methodologies will provide a framework for identifying genetic-environmental-sociobehavioral interactions in depressive symptoms.

A human pluripotent stem cell platform for assessing developmental neural toxicity screening.

A lack of affordable and effective testing and screening procedures mean surprisingly little is known about the health hazards of many of the tens of thousands of chemicals in use in the world today. The recent rise in the number of children affected by neurological disorders such as autism has stirred valuable debate about the role chemicals play in our daily life, highlighting the need for improved methods of assessing chemicals for developmental neural toxicity.

Unachievable Region in Precision-Recall Space and Its Effect on Empirical Evaluation.

Precision-recall (PR) curves and the areas under them are widely used to summarize machine learning results, especially for data sets exhibiting class skew. They are often used analogously to ROC curves and the area under ROC curves. It is known that PR curves vary as class skew changes. What was not recognized before this paper is that there is a region of PR space that is completely unachievable, and the size of this region depends only on the skew. This paper precisely characterizes the size of that region and discusses its implications for empirical evaluation methodology in machine learning.

Predicting atrial fibrillation and flutter using electronic health records.

Electronic Health Records (EHR) contain large amounts of useful information that could potentially be used for building models for predicting onset of diseases. In this study, we have investigated the use of free-text and coded data in Marshfield Clinic's EHR, individually and in combination for building machine learning based models to predict the first ever episode of atrial fibrillation and/or atrial flutter (AFF). We trained and evaluated our AFF models on the EHR data across different time intervals (1, 3, 5 and all years) prior to first documented onset of AFF. We applied several machine learning methods, including naïve bayes, support vector machines (SVM), logistic regression and random forests for building AFF prediction models and evaluated these using 10-fold cross-validation approach. On text-based datasets, the best model achieved an F-measure of 60.1%, when applied exclusively to coded data. The combination of textual and coded data achieved comparable performance. The study results attest to the relative merit of utilizing textual data to complement the use of coded data for disease onset prediction modeling.

Probabilistic computer model developed from clinical data in national mammography database format to classify mammographic findings.

PURPOSE: To determine whether a Bayesian network trained on a large database of patient demographic risk factors and radiologist-observed findings from consecutive clinical mammography examinations can exceed radiologist performance in the classification of mammographic findings as benign or malignant. MATERIALS AND METHODS: The institutional review board exempted this HIPAA-compliant retrospective study from requiring informed consent. Structured reports from 48 744 consecutive pooled screening and diagnostic mammography examinations in 18 269 patients from April 5, 1999 to February 9, 2004 were collected. Mammographic findings were matched with a state cancer registry, which served as the reference standard. By using 10-fold cross validation, the Bayesian network was tested and trained to estimate breast cancer risk by using demographic risk factors (age, family and personal history of breast cancer, and use of hormone replacement therapy) and mammographic findings recorded in the Breast Imaging Reporting and Data System lexicon. The performance of radiologists compared with the Bayesian network was evaluated by using area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. RESULTS: The Bayesian network significantly exceeded the performance of interpreting radiologists in terms of AUC (0.960 vs 0.939, P = .002), sensitivity (90.0% vs 85.3%, P < .001), and specificity (93.0% vs 88.1%, P < .001). CONCLUSION: On the basis of prospectively collected variables, the evaluated Bayesian network can predict the probability of breast cancer and exceed interpreting radiologist performance. Bayesian networks may help radiologists improve mammographic interpretation.

Prion disease diagnosis by proteomic profiling.

Definitive prion disease diagnosis is currently limited to postmortem assay for the presence of the disease-associated proteinase K-resistant prion protein. Using cerebrospinal fluid (CSF) from prion-infected hamsters, matrix-assisted laser desorption/ionization Fourier transform mass spectrometry (MALDI-FTMS), and support vector machines (SVM), we have identified peptide profiles characteristic of disease state. Using 10-fold leave-one-out cross-validation, we report a predictive accuracy of 72% with a true positive rate of 73% and a false positive rate of 27% demonstrating the suitability of using proteomic profiling and CSF for the development of multiple marker diagnostics of prion disease.

Evaluation of multiple models to distinguish closely related forms of disease using DNA microarray data: an application to multiple myeloma.

MOTIVATION: Standard laboratory classification of the plasma cell dyscrasia monoclonal gammopathy of undetermined significance (MGUS) and the overt plasma cell neoplasm multiple myeloma (MM) is quite accurate, yet, for the most part, biologically uninformative. Most, if not all, cancers are caused by inherited or acquired genetic mutations that manifest themselves in altered gene expression patterns in the clonally related cancer cells. Microarray technology allows for qualitative and quantitative measurements of the expression levels of thousands of genes simultaneously, and it has now been used both to classify cancers that are morphologically indistinguishable and to predict response to therapy. It is anticipated that this information can also be used to develop molecular diagnostic models and to provide insight into mechanisms of disease progression, e.g., transition from healthy to benign hyperplasia or conversion of a benign hyperplasia to overt malignancy. However, standard data analysis techniques are not trivial to employ on these large data sets. Methodology designed to handle large data sets (or modified to do so) is needed to access the vital information contained in the genetic samples, which in turn can be used to develop more robust and accurate methods of clinical diagnostics and prognostics. RESULTS: Here we report on the application of a panel of statistical and data mining methodologies to classify groups of samples based on expression of 12,000 genes derived from a high density oligonucleotide microarray analysis of highly purified plasma cells from newly diagnosed MM, MGUS, and normal healthy donors. The three groups of samples are each tested against each other. The methods are found to be similar in their ability to predict group membership; all do quite well at predicting MM vs. normal and MGUS vs. normal. However, no method appears to be able to distinguish explicitly the genetic mechanisms between MM and MGUS. We believe this might be due to the lack of genetic differences between these two conditions, and may not be due to the failure of the models. We report the prediction errors for each of the models and each of the methods. Additionally, we report ROC curves for the results on group prediction. AVAILABILITY: Logistic regression: standard software, available, for example in SAS. Decision trees and boosted trees: C5.0 from www.rulequest.com. SVM: SVM-light is publicly available from svmlight.joachims.org. Naïve Bayes and ensemble of voters are publicly available from www.biostat.wisc.edu/~mwaddell/eov.html. Nearest Shrunken Centroids is publicly available from http://www-stat.stanford.edu/~tibs/PAM.